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1.  Summary  of  Phase  I  Research  Results 


All  Phase  I  research  objectives  were  met  or  even  exceeded.  An  integrated  prototype  software 
application  was  created  based  on  the  theoretical  results  developed  during  the  course  of  the 
project.  While  using  this  software  environment,  members  of  our  team  became  the  first 
humans  to  “feel”  the  interacting  forces  between  micro  and  nano  scale  objects. 

During  the  course  of  the  project,  it  became  clear  that  the  computational  burden  involved  in 
the  exact  modeling  of  the  relevant  physical  forces  constitutes  a  major  challenge.  Clearly,  the 
introduction  of  more  and  more  powerful  computers  will  continuously  extend  the  realm  of 
solvable  problems.  Quite  naturally,  however,  the  user  will  always  have  the  desire  to  take  his 
application  to  the  next  level  and  treat  an  even  more  elaborate  problem.  Several  ideas  to 
further  speed  up  the  calculation  of  the  physical  forces  were  developed  during  the  Phase  I 
research.  These  ideas  will  be  further  investigated  in  Phase  II  and  implemented  in  a 
production  software  environment. 


2.  Discussion  of  Phase  I  Research  Results 


In  Sections  2.1  -2.8,  the  findings  of  the  tasks  outlined  in  the  Phase  I  proposal  are  discussed. 

2.1  Task  1:  Evaluate  the  state  of  the  art  in  VR-based  virtual  assembly  and  understand  the 
issues  confronting  the  deployment  of  virtual  assembly  for  IPPD 


The  following  is  a  summary  of  the  various  leading  activities  in  VR-based  virtual  assembly. 

Virtual  Assembly  Design  Environment  (VADE)  [Jayaram  1999]  has  been  designed  and 
implemented  at  Washington  State  University  in  collaboration  with  NIST.  This  system 
supports  several  types  of  physically-based  modeling  methods  to  allow  engineers  to  simulate 
interactively  the  assembly  of  medium  and  large  sized  mechanical  systems  (automotive, 
heavy  machinery,  etc.).  Archimedes  [Kaufmanl996]  is  an  assembly  planning  research 
project  in  the  Intelligent  Systems  and  Robotics  Center,  or  ISRC,  at  Sandia  National 
Laboratories.  Archimedes  which  creates  an  assembly  plan  based  on  a  CAD  assembly. 
Engineering  Animation  Incorporated  (EAI)[WWW02]  markets  products  that  model  and 
simulate  manufacturing  and  production  line  processes  including  assembly.  The  focus  of 
EAI's  work  pertaining  to  virtual  assembly  has  been  on  simulating  and  evaluating 
dimensional  variation  issues  in  product  assembly.  Work  at  Heriot-Watt  University 
[WWW05]  in  the  field  of  virtual  assembly  has  focused  on  two  areas.  The  first  is  the  use  of 
immersive  virtual  reality  to  design  and  plan  cable  harness  layouts.  The  second  focus  of 
virtual  assembly  at  Heriot-Watt  University  is  the  use  of  virtual  reality  to  extract  elicit 
knowledge  from  the  user  for  assembly  planning. 
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Work  at  Carnegie  Mellon  University  developed  into  an  "Intelligent  Assembly  Modeling  and 
Simulation"  environment  (I AMS)  [Gupta  1997].  IAMS  is  an  intelligent  environment  in 
which  simple  simulation  tools  can  be  composed  into  complex  simulations  for  detecting 
potential  assembly  problems.  Deneb  Robotics,  Inc.,  has  produced  a  product  called 
Deneb/Assembly  [WWW04]  to  aid  in  Design  For  Assembly  tasks.  Deneb/Assembly  is  a 
engineering  tool  used  to  capture  part  paths,  detect  collisions,  generate  collision-free  paths, 
inspect  internal  parts  and  clearances,  simulate  cable  and  wire  harnesses,  and  generate  shop 
floor  instructions.  Research  at  the  University  of  North  Carolina  in  Chapel  Hill,  North 
Carolina,  has  been  focused  on  a  project  called  the  nanoManipulator  [WWW03].  The 
researchers  are  developing  an  improved,  natural  interface  to  scanning  probe  microscopes, 
including  Scanning  Tunneling  Microscopes  (STM)  and  Atomic  Force  Microscopes  (AFM). 
The  nanoManipulator  (nM)  couples  the  microscope  to  a  virtual-reality  interface  to  provide  a 
telepresence  system  that  operates  over  a  scale  difference  of  about  a  million  to  one.  The 
nanoManipulator  has  been  used  to  examine  viruses  and  other  micro  scale  structures. 

2.2  Task  2; — Evaluate,  existing  methods  for  simulating  the  forces-experienced  during-the 

assembly  of  In2m  devices 


The  relative  importance  of  external  forces  on  a  MEMS  device  is  very  different  from  that  in 
the  macro  world.  Unlike  with  traditionally  sized  parts,  gravity  is  not  the  dominant  force 
acting  on  a  MEMS  device.  As  components  become  smaller  than  1  millimeter,  gravity  is 
almost  negligible,  while  surface  adhesion,  electrostatic,  electromagnetics,  and  van  der  Waals 
forces  dominate  (See  Figure  1).  These  forces  are  visible  during  the  fabrication,  assembly, 
and  operation  of  all  MEMS  devices.  During  wet  etching  operations,  these  forces  cause 
freed  structures  to  be  attracted  to  one  another.  During  the  assembly  of  hybrid  MEMS,  these 
forces  cause  MEMS  components  to  adhere  to  tools.  During  operation  of  MEMS  devices, 
these  forces  act  as  highly  nonlinear  friction  terms. _ 


Figure  1 .  Forces  of  attraction  between  a  sphere  of  a  given  radius  and  a  plane. 
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In  the  research  literature,  most  of  the  work  in  this  area  has  been  performed  by  particle 
physicists  who  are  interested  in  the  adhesion  of  sprayed  (often  spherical)  particles  onto  flat 
objects.  For  forces  such  as  van  der  Waals,  there  are  closed  form  analytical  solutions  for  very 
simple  geometrically  shaped  objects  such  as  the  interaction  between  two  spheres  or  a  sphere 
and  a  half  space.  However,  finite  element  codes  that  predict  the  van  der  Waals  force 
between  arbitrarily  shaped  objects  do  not  exist.  Similarly,  there  exist  formulas  for  surface 
tension  between  simple  shaped  hydrophilic  surfaces,  but  not  for  complex  shapes.  In  the  area 
of  electrostatics,  several  finite  and  boundary  element  codes  exist.  However,  these  codes  are 
typically  used  to  estimate  the  capacitance  between  objects,  and  not  the  interfacial  forces. 

Within  Phase  I  of  this  project,  we  modified  a  Sandia-developed  boundary  element 
electrostatics  code,  called  EIGER_S,  to  calculate  the  forces  of  interaction  between  arbitrarily 
shaped  metal  parts.  Solid  models  of  the  parts  where  created  in  SolidWorks,  and  meshed 
surfaces  were  generated  using  the  commercial  package  Ideas.  The  virtual  reality 
environment  displayed  the  solid  model  from  the  SolidWorks,  while  the  EIGER  S  code 
determined  the  forces  acting  on  the  meshed  surfaces. 


2.3  Task  3:  Evaluate  the  level  of  physically-based  modeling  required  to  adequately  simulate 
the  interactions  between  objects 


The  interactions  between  objects  during  the  assembly  of  In2m  devices  can  be  broken  down 
into  two  types:  a)  forces  of  interaction  when  the  objects  are  close  to  each  other,  and  b)  forces 
of  interaction  when  the  objects  touch  or  collide  with  each  other.  Both  types  of  forces  are 
important  in  simulating  these  assemblies  and  the  forces  in  the  transition  zone  are  particularly 
difficult  to  simulate.  This  phase  of  the  project  focused  on  the  electrostatic  forces  between 
objects  and  the  forces  of  collision. 

The  process  of  solving  for  and  displaying  the  electrostatic  forces  in  MEMS  assembly 
involves  four  steps.  The  first  step  is  to  obtain  a  boundary  element  solution  for  the 
electrostatic  forces.  This  involves  obtaining  a  solution  at  multiple  points,  which  maps  out  a 
working  volume.  Currently,  forces  and  moments  are  calculated  throughout  this  working 
volume  for  the  electrostatic  interaction  of  the  MEMS  component  (work  piece)  and  the 
tweezers  (tool)  used  in  the  assembly  process.  The  solution  is  obtained  at  each  point  in  the 
working  volume  for  multiple  orientations  of  the  work  piece  and  tool.  The  combination  of 
these  orientation  and  position  solutions  creates  a  full  six  degrees  of  freedom  solution  grid. 
The  computational  time  required  for  each  calculation  prevents  the  solutions  from  being 
calculated  and  displayed  at  or  near  real  time.  Therefore,  the  second  step  in  the  process 
involves  performing  a  real-time  interpolation  of  the  electrostatic  forces  based  on  the  current 
relative  position  and  ori  entation  of  the  tool  to  the  work  piece.  This  interpolation  provides  us 
with  an  approximation  of  the  forces  and  moments  as  we  move  through  the  virtual  MEMS 
assembly  environment.  The  third  step  of  the  process  is  the  application  of  a  scaling  factor  to 
the  forces  and  moments  in  order  to  display  them  to  the  user  at  a  level  that  can  be  felt  and 
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interpreted.  The  fourth  step  of  the  process  is  to  perform  a  force  analysis  of  the  system  to 
decide  if  the  electrostatic  forces  of  attraction  are  great  enough  to  cause  the  work  piece  to 
move  towards  or  stick  to  the  tool  or  any  other  object  in  the  environment.  When  the  work 
piece  is  attached  to  the  tool,  the  tool  is  allowed  to  move  the  work  piece  throughout  the 
environment  until  an  analysis  of  the  forces  acting  on  the  work  piece  causes  it  to  free  itself 
from  the  tool.  Figure  2  shows  a  typical  tweezer/work  piece  combination  for  MEMS 
assembly. 


Figure  2.  Moving  tweezers  close  to  the  micro-gear. 

A  boundary  element  electrostatic  program  called  EIGERJS  is  being  used  to  solve  for  the 
micro-scale  forces  and  moments  between  two  bodies  (a  gripper  and  the  part  to  be  grasped). 
This  software  was  originally  developed  by  Sandia  National  Laboratories  to  evaluate  the 
electrostatic  fields  on  various  weapons  components.  In  this  project,  we  have  added  the 
capability  to  compute  the  electrostatic  forces  and  moments  between  multiple  bodies. 


2.4  Task  4:  Investigate  methods  forreab-time-  computation  of  .interaction  forces  for  display  in 
the  virtual  environment 

A  promising  approach  to  speed  up  the  calculation  of  the  electrostatic  forces  and  moments  was 
developed.  This  approach  is  based  on  an  idea  that  enables  a  computationally  inexpensive  calculation 
of  the  sensitivities  of  the  electrostatic  field  with  respect  to  changes  in  the  configuration.  These 
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sensitivities  are  then  used  in  a  linear  extrapolation  scheme.  Details  of  this  approach  are  discussed  in 
the  AMA  report  [Seywald,  2000]. 


2.5  Task  5:  Create  methods  to  “display”  the  forces  of  interaction  using  multi-modal  VR 
environments 


The  solutions  of  the  boundary  value  equations  give  us  forces  on  the  order  of  lO  ^N. 
Obviously,  forces  this  small  are  not  naturally  felt  by  humans.  Therefore,  a  scaling  factor 
needs  to  be  used  to  scale  these  forces  up  to  human  scale.  When  we  do  this  we  must  keep  in 
mind  that  there  are  two  types  of  forces  that  we  will  feel  as  users  of  this  system.  The  first 
type  of  force  is  the  electrostatic  force  that  is  calculated  by  the  boundary  element  solution,  the 
second  type  of  force  is  the  contact  force  of  our  tool  colliding  with  another  object  in  the 
environment.  Contact  forces  are  reaction  forces  and  are  determined  by  using  friction  factors 
and  force  vector  balances.  Contact  forces  between  the  tool  and  a  MEMS  component  also 
need  to  be  scaled  in  order  to  achieve  a  force  resolution  that  we,  as  human  users,  are  capable 
of  detecting.  When  determining  what  the  scaling  factor  must  be,  we  have  to  keep  in  mind 
that  there  sometimes  needs  to  be  a  transition  from  electrostatic  forces  to  contact  forces.  This 
is  especially  true  since  the  electrostatic  forces  are  discharged  to  zero  when  the  two  objects 
come  in  contact. 

SensAble  Technologies'  PHANToM  device  provides  three  degree  of  freedom  force  feedback 
to  a  user  through  the  use  of  servo  motors  and  lever  arms  (Figure  3). 


Figure  3.  Schematic  of  the  PHANToM  device 

In  a  typical  application,  the  PHANToM  is  used  to  display  contact  forces  between  two 
physical  objects.  Collision  detection  is  used  to  determine  when  objects  are  in  contact.  The 
corresponding  reaction  force  vector  is  calculated  and  then  applied.  The  PHANToM  device 
supports  six  degrees  of  freedom  as  an  input  device  (position  and  orientation)  and  three 
degrees  of  freedom  as  an  output  device  (force  vector).  In  the  case  of  the  electrostatic  forces, 
we  have  a  force  field  where  the  relative  position,  orientation  and  potential  difference 
between  two  or  more  objects  create  an  attractive  force  to  be  displayed  by  the  PHANToM. 


8 


tv 


These  forces  can  easily  be  applied  using  the  PHANToM  by  integrating  the  interpolation 
code  to  look  up  the  necessary  force  vector  for  current  position  of  the  tool.  When  the 
assembly  tool  come  into  contact  with  the  MEMS  component,  the  potential  difference 
between  the  two  object  is  zero,  thus,  the  electrostatic  forces  between  the  two  objects  are 
zero.  At  this  point  we  are  no  longer  using  a  force  field.  Instead,  the  reaction  force  due  to  the 
contact  of  the  tool  with  the  MEMS  component  is  the  force  displayed  with  the  PHANToM 
(Figure  4). 


Figure  4.  Choosing  the  type  of  force  to  be  displayed 

Several  generic  methods  were  created  to  use  the  Phantom  and  the  CyberGrasp  (from  Virtual 
Technologies)  to  display  various  types  of  forces.  These  methods  were  used  to  create  the 
integrated  prototype  application  in  task  8. 

2.6-Task  6: _ Create  a  “virtual  microscope”  to  allow  a  true  evaluation  of  whatihe-assembly 

operator  or  assembly  robot  will  “see”  during  the  real  assembly  process 


A  virtual  microscope  was  created  using  a  pair  of  Tektronix  EXD-100  display  tubes  and  the 
associated  optics  (Figure  5).  In  order  to  recreate  the  realism  of  a  microscope,  two  different 
approaches  were  used  to  “de-focus”  the  image.  The  low  depth  of  field  and  the  image  blur 
rendered  a  realistic  virtual  microscope  image  which  was  integrated  into  the  final  application. 
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Figure  5.  Using  the  PHANToM  and  the  virtual  microscope  to  manipulate  the  tweezers 

2.7  Task  7:  Evaluate  the  data  requirements  for  the  physically-based  modeling  of  In2m 
assembly  process  and  investigate  if  these  data  can  be  obtained  from  CAD  systems 


The  data  requirements  for  the  calculation  and  display  of  forces  of  interaction  in  In2m 
assembly  are  primarily  geometric.  The  surface  geometry  of  the  object  is  required  for  the 
electrostatic  force  calculations  and  for  the  detection  of  proximity  and  collision.  The  center 
of  mass,  moments  of  inertia,  and  other  volumetric  properties  are  required  for  simulating  the 
response  of  the  object  subject  to  these  forces.  All  of  the  geometric  and  volumetric 
information  can  be  extracted  from  most  CAD  systems.  The  VADE  application 
automatically  extracts  all  of  this  information  from  the  CAD  system  for  simulating  gravity 
induced  motion. 

2.8  Task  8:  Integrate  all  of  the  above  tasks  into  a  proof-of-concept  application 


An  integrated  prototype  application  was  created  based  on  the  above-described  tasks.  This 
application  allows  a  user  to  view  the  assembly  process  in  the  virtual  microscope  while 
manipulating  a  part  with  a  tweezer.  Haptic  feedback  is  provided  through  the  Phantom.  The 
electrostatic  forces  are  pre-computed  for  a  region  and  interpolation  techniques  are  used  to 
display  the  real-time  forces  during  assembly. 
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